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APPARATUS AND METHOD FOR ROTATING factured with the Bayer pixel pattern format will be coupled 

THE DISPLAY ORIENTATION OF A to image processing routines specifically designed to accept 

PAPTURFD IMAGE P ixel s 'S nals 10 alternating sequences ol GK(jK(jK ana 

"BGBGBG." Due to possible imperfections in the outer 

This application relates to co-pending U.S. patent appli- 5 rows and columns of pixels that make up the image sensor, 

cation SenNo. 08/355,031, entitled A System and Method conventtona digital cameras somet.mes have m.age senso* 

For Generating a Contrast 6verlay as a Focus Assist for An large enough so that one or more lines of pixels at the sides 

Imaging Device, filed on Dec. 13, 1994, by inventor Eric C. of the image sensor can be ignored. 

Anderson; and U.S. patent application Ser. No. 08/384,012, Referring now to FIG. ID, a pnor art graphical represen- 

entitled Apparatus and Method for Camera Image and w tation is shown of the object as captured m the portrait 

Orientation Capture, filed on Feb. 6, 1995, by inventor Scott orientation and output upon an image cusp ay 100. The 

Fullam. The subject matter of the two applications described image display 100 « typically a conventional stand-alone, 

above is hereby incorporated by reference. These related personal computer CRT having a top 102, a right side 104, 

applications are commonly assigned to Apple Computer, a left side 106 and a bottom 108. 

I™ ' 15 As previously described, the image processing routines 

within digital cameras are conventionally designed to pro- 

BACKGROUND OF THE INVENTION C ess pixel signals on a line-by-line basis according to only 

1 Field of the Invention one P ixeI P attern format - Thus ' conventional di § ital cameras 

process images as if they were always in a landscape format. 

TTie present invention relates generally tc . an waratus ^ £ tfa eprese L tion of the landscape image of FIG. 1A upon 

and method for orienting an mage More particularly, the * ^ „ TOp>f e ^ Qf ^ 

present invention is an apparatus and method for rotating a knds * j wodd correspond to the top 102 of the 

captured image to an orientation i corresponding to . an imag- * m Such ^ orientation for land . 

ing subsystem's orientation at the time in which the image ^ ^ ^ display 10Q fe quite natural and 

was captured. 25 ig acceptaD i e for ease of viewing the image. However, the 

2. Description of the Background Art presentation of the portrait image of FIG. IB upon the image 

When a digital camera captures an image of an object, the display results in the "TOF' of the portrait image corre- 

camera's frame of reference with respect to the object sponding to either the right side 104 or the left side 106 of 

produces a desired image orientation. Two conventional me image display ioo, depending on how the user had 

image orientations exist, namely, a landscape orientation and 30 rotated tne digital camera. FIG. ID explicitly shows the case 

a portrait orientation. Referring now to FIG. 1A, a prior art where the "TOP" of the portrait image corresponds to the 

graphical representation of an object in a landscape orien- rigm side ^ of tne i mage display 100. This "sideways" 

tation is shown, in which the image's width is greater than orientation for portrait images on the image display 100 is 

its height. Referring also now to FIG. IB, a prior art unacceptable and unnatural This undesirable by-product of 

graphical representation of the object in a portrait orientation 35 conV entional digital cameras requires that the user rotate 

is shown, in which the image's height is greater than its portrait images so that they are presented in a more natural 

width. and upright viewing angle on the image display 100. It 

In a digital camera, an image sensor is comprised of should be noted that this problem also exists for digitized 

light-sensitive devices, such as charge-coupled devices images from conventional film cameras. Traditional rotation 

(CCD), that convert an optical image into a set of electrical 40 methods also have the disadvantage of requiring two blocks 

signals. Referring now to FIG. 1C, a prior art image sensor 0 f memory to rotate a stored image, 

is shown having a 480 row by 640 column matrix of light What ^ need ed is an apparatus and method that efficiently 

collecting pixels. The image sensor is orientated within a and automatically rotates a stored photographic image to 

body of the digital camera so that a first row of pixels (i.e. correspond to the orientation in which the photographic 

row r x ) corresponds to the bottom of an upright and level 45 image was captured. 

digital camera. This image sensor orientation is required " SUMMARY OF THE INVENTION 

since as an optical image passes through a conventional SUMMARY Or- IHb iwvumiuw 

camera lens it is inverted. The image sensor in a color digital The present invention is an apparatus and method for 

camera typically consists of an array of green (G), red (R) rotating the display orientation of a captured image. The 

and blue (B) pixels. Alternative embodiments include sen- 50 apparatus of the present invention preferably comprises an 

sors detecting cyan, magenta, yellow and green as is typi- image sensor, an orientation sensor, a memory and a pro- 

cally used in video cameras. Other image sensor configu- cessing unit. The image sensor is used for capturing image 

rations are also used. The pixels that comprise the image data. The orientation sensor is coupled to the image sensor, 

sensor are arranged into various patterns or formats. A and is used for generating a portrait signal if the image 

common image sensor format is called a Bayer pattern. The 55 sensor is positioned in a portrait orientation relative to the 

Bayer pattern format is defined as a pixel pattern comprised object. The memory, has an auto-rotate unit comprising 

of 50% green-light responsive pixels, 25% red-light respon- program instructions for transforming the captured image 

sive pixels and 25% blue-light responsive pixels arranged in data into rotated image data in response to the portrait signal, 

alternating rows of "GRGRGR" and "BGBGBG," as shown The processing unit, is used for executing program instruc- 

in FIG. 1C. Throughout this specification, "G" means 60 tions stored in the memory, and is coupled to the image 

"green," "R" means "red," and "B" means "blue." sensor, the orientation sensor and the memory. 

Once an image is captured by the digital camera, a set of The method of the present invention preferably comprises 

pixel signals corresponding to the image received by the the steps of: generating image data representative of an 

pixels is processed by an image processing algorithm. Image object with an image sensor; identifying an orientation of the 

processing routines are conventionally designed to process 65 image sensor relative to the object during the generating 

pixel signals line-by-line, conforming to a specific and step; and selectively transferring the image data to an image 

unchanging pixel pattern format. Thus, image sensors manu- processing unit in response to the identifying step. 
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BRIEF DESCRIPTION OF THE DRAWINGS Referring now to FIG. 3A, a block diagram is shown 

illustrating the imaging subsystem 204 of the present inven- 

FIG. 1A, is a prior art graphical representation of an t j on ^he imaging subsystem 204 preferably comprises a 

object in a landscape orientation; lens 300 having an iris, an optical filter 302, an image sensor 

FIG IB is a prior art graphical representation of the s 304, a timing generator 306, an analog signal processor 

object in a 'portrait orientation; * (ASP) 308, an analog-to-digital (A/D) converter 310, a 

FIG. 1C, is a prior art 480 row by 640 column image digital signal [processor (DSP) 320, a n&ttteft^*ton 

* ' ¥ ■> » sensor (RLOS) 324, an up/down orientation sensor (UDOS) 

sensor; 326 and one or more motors 312. U.S. patent application Ser. 

FIG. ID, is a prior art graphical representation of the Nq QS/355fi3U entitled A System and Method For Gener- 

object as captured in the portrait orientation and output upon ^ a Contrast 0verky ^ a Focus For ^ Imaging 

an image display; Device, filed on Dec. 13, 1994 is incorporated herein by 

FIG. 2, is a block diagram of a preferred embodiment of reference and provides a detailed discussion of the preferred 

an apparatus for rotating the display orientation of a cap- elements in the imaging subsystem 204, with the exception 

tured image; of the RLOS 324 and the UDOS 326. U.S. patent application 

FIG. 3A, is a block diagram of a preferred embodiment of Ser. No. 08/384,012, entitled Apparatus and Method for 

an imaging subsystem of the present invention; Camera image and Orientation Capture, filed on Feb. 6, 

FIG 3B, is a first graphical representation of a preferred 1995 is incorporated herein by reference and provides a 

embodiment of an image sensor disposed within the present detailed discussion of various embodiments of the RLOS 

invention- 20 324 and the UD0S 326 > which are further discussed below - 

ferrcd embodo...! of U» ..f *nso, Mho the present ^ ^ M * ^ 3^ fa ,„ g ^,„ a , 

invention, r setof pixel signals representative of the captured image data. 

FIG. 4A, is a block diagram of a preferred embodiment of ^ beiflg processed by the ASP 308j the a/d converter 

a processing and storage subsystem of the present invention; 2 > 31Q and {hQ Dsp 32Q; the captured image data passes over 

FIG. 4B, is a block diagram of a preferred embodiment of me bus 206 to the processing and storage subsystem 208. 

a volatile memory within the processing and storage sub- y^e functions performed by the DSP 320 may alternatively 

system; be performed in the processing and storage subsystem 208. 

FIG. 4C, is a block diagram of a preferred embodiment of 3Q Referring now to FIG. 3B, a graphical representation is 

a frame buffer data structure within a frame buffer; shown illustrating the image sensor 304 disposed within the 

FIG. 4D, is a block diagram of the preferred embodiment camera 200. FIG. 3B shows the camera 200 maintained in a 

of a non-volatile memory within the processing and storage landscape orientation while directly facing the object 202. 

subsystem- Tne image sensor 304 has a front 338 and a top 336, which 

FIG. 5A, is a graphical representation of a preferred 35 are juxtaposed with a front 334 and a bottom 332 of the 

embodiment of portraitjeft image data within the frame camera 200 respective y The front side 338 of the image 

buffer data strucLe; ^ 304 ^t-sensitive pixels, which capture 

^ _ . . . , ... r * A an image received along the optical path 314 and through the 

FIG. 5B, 1S a graphical representation of a preferred ^ ^ ^ camera 200 ft be noted 

embodiment of portrait_nght image data within the frame ^ ^ by ^ image 3Q4 fc ^ 

buffer data structure; relative tQ the object by the lens 300 Xhe front 338 of the 

FIG. 5C, is a graphical representation of a preferred image sensor 304 pre ferably forms a planar surface as 

embodiment of landscape image data within the frame buffer s hown in perspective in FIG. 3B. When the image sensor 

data structure; and 3 Q4 ^ oriented as shown in FIG. 3B, a landscape image of 

FIGS. 6A, 6B, 6C and 6D, are a flowchart of a preferred 45 the object 202 is captured. When the camera 200 is rotated 

method for rotating the display orientation of a captured either 90° clockwise or 90° counterclockwise about the 

image. x-axis, a portrait image of the object 202 is captured. 

twtatt rn nF^PRTPTTOM of THF Referring now to FIG. 3C, a graphical representation is 

'^n^ shown illustrating the image sensor 304, which has the top 

PREFERRED EMBODIMENTS ^ m a fight side %^ a M{ gide 34Q and a bottom 344 ^ 

The present invention is an apparatus and method for image sensor 304 is comprised of a plurality of pixels 

rotating the display orientation of a captured image. arranged in an i-row by j -column array. In the preferred 

Referring now to FIG. 2, a block diagram is shown embodiment, the pixels are arranged in a Bayer pattern 
illustrating a camera 200 for rotating a display orientation of format. Alternatively the present invention may be applied 
a captured image. In the preferred embodiment, the camera 55 to image sensors of various sizes and image processing 
200 is a digital camera, such as a QuickTake 150, by Apple routines designed to process pixel signals arranged in pat- 
Computer of Cupertino, Calif. The camera 200 preferably terns or formats different from the Bayer pattern, 
comprises an imaging subsystem 204, a bus 206 and a As stated above, the RLOS 324 (i.e. the right/left orien- 
processing and storage subsystem 208. Reflected light from tation sensor) and the UDOS 326 (i.e. the up/down orien- 
an object 202 is conventionally captured by the imaging 60 tation sensor) are disposed within the imaging subsystem 
subsystem 204 via a button press or some other action. The 204. Both the RLOS 324 and the UDOS 326 are physically 
captured image data is transferred over the bus 206 to the coupled to the image sensor 304 and electrically coupled to 
processing and storage subsystem 208, which stores the data the DSP 320. The RLOS 324 monitors whether the camera 
in internal memory. The bus 206 also passes various status 200 is in a landscape orientation or a portrait orientation. If 
and control signals between the imaging subsystem 204 and 65 the camera 200 is in a portrait orientation, the RLOS 324 
the processing and storage subsystem 208, as will be further identifies whether the camera 200 has been rotated clock- 
discussed below. wise or counterclockwise, as viewed from the rear of the 
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camera 200. In response to such camera 200 rotation, the The buffer 400 is preferably a first-in, first-out buffer for 
RLOS 324 generates and transmits either a landscape signal, temporarily holding the captured image data and the imag- 
a portrait_left signal or a portrait_right signal to the DSP ing subsystem 204 control signals passed between the DSP 
320. More specifically, if the camera 200 is held in the 320 and the processing and storage subsystem 208. The 
landscape orientation shown in FIGS. 1A and 3B, the RLOS 5 buffer 400 has data lines that are coupled to both the bus 206 
324 generates the landscape signal, since the top 336 of the and the internal bus 402. The processing unit 404 executes 
image sensor 304 corresponds to the "TOP" of the object programming instructions stored in the non-volatile memory 
202 & However, if the camera 200 is rotated 90° clockwise 406 and the volatile memory 410 to perform various opera- 
about the x-axis, as viewed from the rear of the camera 200 tions. The non-volatile memory 406 stores a set of computer 
shown in FIG. 3B, the RLOS 324 generates the portrait_ 10 readable program instructions for controlling how the pro- 
right signal, siuce now the right side 342 of the image sensor cessing unit 404 accesses, transforms and outputs the image 
304 corresponds to the "TOP" of the object 202. And, if the data, as described in detail below with reference to FIG. 4D. 
camera 200 is rotated 90° counter-clockwise about the While the non-volatile memory 406 is preferably a Read 
x-axis, as viewed from the rear of the camera 200, the RLOS Only Memory (ROM), alternatively the non-volatile 
324 generates the portrait_left signal, since now the left side 15 memory 406 could be replaced with a functionally equiva- 
340 of the image sensor 304 corresponds to the "TOP" of the lent computer useable medium such as: a compact disk and 
object 202. drive; a floppy disk and drive; or a memory card. Additional 

The UDOS 326 monitors whether the camera 200 is in details on the function and operation of both the non-volatile 

either a level orientation or an off-level orientation. In memory 406 and the volatile memory 410 are provided 

response to a movement of the camera 200, the UDOS 324 20 below with reference to FIGS. 4B and 4D. 

generates and transmits either a level signal or an off-level The input device 408 is preferably a series of control 

signal to the DSP 320. Specifically, if the camera 200 is held buttons, responsive to user inputs, which generates signals 

in the landscape orientation shown in FIG. 3B, the UDOS that are translated by the processing unit 404 into control 

324 generates the level signal, since the front 334 of the signals to adjust the focus and other operating parameters of 

camera 200 directly faces the object 202. If the camera 200 25 the imaging subsystem 204. As mentioned above, while the 

is rotated more than 45° clockwise or counter-clockwise preferred embodiment of the camera 200 incorporates the 

about the y-axis, shown in FIG. 3B, the UDOS 324 prefer- RLOS 324 and UDOS 326, an alternative embodiment of the 

ably generates the off-level signal, since the front 334 of the present invention prompts the user to manually input, via the 

camera 200 now points either above or below the x-y plane. input device 408, the imaging subsystem's 204 orientation at 

Alternative embodiments of the present invention can 30 the time the captured image data was generated. Other 

change the specific measure of rotation necessary to gener- alternative embodiments incorporating the automatic RLOS 

ate the off-level signal from 45° to any number from 0° to 324 and UDOS 326 sensors could provide for an override 

180° function that instructs the processing and storage subsystem 

Both the RLOS 324 and UDOS 326 sensors are preferably 208 to ignore the orientation signals generated by the RLOS 

identical in their structure and function. Their placement 35 324 and UDOS 326 sensors, enabling the user to retain full 

relative to the image sensor 304 determines whether the manual control over the final orientation of the captured 

signal generated by the orientation sensor is one of the image data on the image display 100. The I/O Interface 414 

RLOS 324 signals or the UDOS 326 signals. The DSP 320 is coupled to the internal bus 402 and has an external port 

receives both the RLOS 324 and the UDOS 326 signals and connector for coupling the processing and storage subsystem 

transmits them to the processing and storage subsystem 208 40 208 with a host computer (not shown) for downloading 

over the bus 206. While the preferred embodiment of the image data stored in the volatile memory 410 and thus 

present invention incorporates the orientation sensors 324, freeing up room in the volatile memory 410 for additional 

326, those skilled in the art will recognize that the orienta- sets of processed image data. In the second and alternative 

tion sensors 324, 326 are not required to practice the present embodiment discussed above where the processing and 

invention if the processing and storage subsystem 208 45 storage subsystem 208 is a desktop type computer, the input 

prompts the photographer for the camera's 200 orientation at device 408 will also include a keyboard and mouse type 

the time an image is captured. Alternatively, an embodiment controller. 

using only the RLOS 324 enables practicing of the present Referring now to FIG. 4B, a block diagram is shown 

invention, but this alternative embodiment results in loss of illustrating a volatile memory 410 within the processing and 

the off-level signal. However, such an embodiment using 50 storage subsystem 208. The volatile memory 410 contains a 

only the RLOS 324 is still useful for most applications frame buffer 416, working memory 418 and storage memory 

requiring only landscape and portrait orientation using an 420 coupled to the internal bus 402. The volatile memory 

approximately level camera 200. A further alternative 410 is preferably a Random Access Memory (RAM), 

embodiment includes an RLOS 324 which only generates The frame buffer 416 is preferably comprised of a dedi- 

the landscape, portrait__left, or portrait_right signals if the 55 cated space of contiguous memory comprising an i-row by 

camera 200 is within a predetermined level range. j-column area for storing image data. There is preferably a 

Referring now to FIG. 4A, a block diagram is shown constant one-to-one correspondence between the pixels 

illustrating the processing and storage subsystem 208 of the comprising the image sensor 304 array and the storage 

present invention. The processing and storage subsystem locations within the frame buffer 416 area. Alternatively, the 

208 comprises a buffer 400, a processing unit 404, a non- 60 pixel signals from the image sensor 304 array could be 

volatile memory 406, an input device 408, a volatile stored within the frame buffer 416 in a non-contiguous (i.e. 

memory 410 and an I/O interface 414, coupled together via sectored) memory format, as is commonly known in the art. 

an internal bus 402. In the preferred embodiment, the In the preferred embodiment, i and j are set equivalent to the 

processing and storage subsystem 208 is conventionally size of the image sensor 304. Since the preferred image 

embedded as part of the camera 200. Alternatively, the 65 sensor 304 is a 481 by 641 array, i is set to 481 and j is set 

processing and storage subsystem 208 may be a desktop to 641. In alternative embodiments where the image sensor 

type computer. 304 is only a 480 by 640 array, the frame buffer 416 will still 
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preferably be a 481 by 641 array since, as is discussed 
below, an extra row and column is generated by the pro- 
cessing and storage subsystem 208. 

The working memory 418 preferably provides a tempo- 
rary space for storing data received from the input device 
408 and for data generated by the processing unit 404. The 
working memory 418 also contains additional memory for 
storing the RLOS 324 and UDOS 326 signals at the time the 
image data was captured by the image sensor 304; and 
additional information conventionally associated with the 
capture process, such as exposure and focus settings, time 
and date, statistical data, and the presence of any defective 
pixels, rows or columns within the image sensor 304. 

The storage memory 420 preferably provides space for 
storing multiple sets of image data, until the images are 
off-loaded from the camera 200 via the I/O interface 414. 
The storage memory 420 sets a maximum image data 
holding capacity of the camera 200. Alternatively, additional 
memory cards could also be coupled to the internal bus 402, 
providing even more image data storage capability. Such an 
additional memory card could be a non-volatile flash 
memory card. 

Referring now to FIG. 4C, a block diagram is shown 
illustrating a memory map of a volatile memory data struc- 
ture 417 within the volatile memory 410. The volatile 
memory data structure 417 comprises an image data field 
419 for storing the captured image data, an RLOS signal 
data field 421 for storing the RLOS 324 signal, an UDOS 
signal data field 423 for storing the UDOS 326 signal and a 
miscellaneous data field 425 for storing the exposure and 
focus settings, time and date, statistical data, and the pres- 
ence of any defective image sensor 304 pixels, rows or 
columns. The interactions of each of all these data fields 419, 
421, 423, 425 with the camera 200 is described in detail 
below. 

Referring now to FIG. 4D, a block diagram is shown 
illustrating the non-volatile memory 406 within the process- 
ing and storage subsystem 208. The non-volatile memory 
406 comprises an auto-rotate unit 422, an image processing 
unit 424, an image capture unit 428 and an operating system 
unit 426. 

The image capture unit 428 controls the imaging sub- 
system's 204 focus and aperture settings; captures the image 
data; routes the image data into the frame buffer 416; routes 
the exposure and focus settings, time and date, statistical 
data, and the presence of any defective image sensor 304 
pixels, rows or columns into the miscellaneous data field 
425; generates additional rows and columns of image data 
within the frame buffer 416 if required; monitors the RLOS 
324 and UDOS 326; and sets the RLOS signal 421 data field 
and the UDOS signal 423 data field based on the RLOS 324 
and UDOS 326 outputs. In the preferred embodiment the 
image capture unit 428 corrects any pixel defects within the 
image sensor 304 using the information stored in the mis- 
cellaneous data field 425 before passing the image data to 
the auto-rotate unit 422. In an alternative embodiment, 
image processing unit 424 corrects any pixel defects within 
the image sensor 304 using the information stored in the 
miscellaneous data field 425 after receiving the image data 
from the auto-rotate unit 422. In this alternative 
embodiment, the defective image sensor 304 pixel informa- 
tion must also be rotated by the auto-rotate unit 422 in the 
same manner as for the image data, described below. 

The image capture unit 428 preferably is designed in two 
embodiments, a hardware embodiment and a software 
embodiment; however, the hardware embodiment is pre- 
ferred. 
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Hardware Embodiment 

In the hardware embodiment, the image sensor 304 is 
comprised of at least one more extra row and column than 
the image processing unit 424 is designed to process. Thus, 

5 if the image processing unit 424 is designed to process a 480 
by 640 pixel signal array, then the image sensor 304 must be 
at least a 481 by 641 pixel array (i.e. "i" is 481 and "j" is 641 
in FIG. 3C). The discussion that follows on the auto-rotate 
unit 422 makes clear when and why the additional row and 

10 column is required. 
Software Embodiment 

The software embodiment is required when the image 
sensor 304 lacks any additional rows and columns beyond 
those that the image processing unit 424 is designed to 

15 process. For Bayer pattern image processing, the image 
capture unit 428 must generate an additional row and 
column of image data. (For non-Bayer pattern image data, a 
straightforward rotation is performed as is described below 
and an additional row and column is not required.) Thus, for 

20 example, if the image sensor 304 is comprised of a 480 by 
640 pixel array and the image processing unit 424 is 
designed to process a 480 by 640 array, then the image 
capture unit 428 must generate an additional row and 
column of pixels. In the preferred software embodiment, the 

25 additional row generated corresponds to row r f - (i.e. 
GRGRGR) in FIG. 3C and the additional column generated 
corresponds to column c, (i.e. GBGBGB) in FIG. 3C. The 
additional row and column may be generated in any number 
of ways, including: setting the additional pixels to black; 

30 creating a duplicate of a nearby line of pixels with the same 
pixel pattern (i.e. two pixel lines over); generating a dupli- 
cate of the nearest color (i.e. G's would be copied from one 
pixel line over while R's or B's would be copied from two 
pixel lines over); setting the pixel values based on a filtering 

35 algorithm which looks at a number of nearby pixels; or 
setting the additional row and column of pixels randomly. 

The auto-rotate unit 422 transforms the captured image 
data into the rotated image data for further processing by the 
image processing unit 424. The image processing unit 424 

40 receives the rotated image data from the auto-rotate unit 422 
and transforms it into processed image data, ready for saving 
in the storage memory 420. The operating system unit 426 
is coupled to the internal bus 402 and preferably comprises 
a conventional set of program instructions for controlling the 

45 configuration and usage of the computer's 208 hardware 
resources. The operations performed by the auto-rotate unit 
422 and the image processing unit 424 are described in more 
detail below. 

Referring now to FIGS. 5A, 5B and 5C, a graphical 
50 representation is shown in FIG. 5A illustrating portrait_left 
image data within the frame buffer 416; a graphical repre- 
sentation is shown in FIG. 5B illustrating portrait__right 
image data within the frame buffer 416; and a graphical 
representation is shown in FIG. 5C illustrating landscape 
55 image data within the frame buffer 416. FIGS. 5A, 5B and 
5C all show the same image data 419 within the frame buffer 
data structure 417 filled with different sets of captured image 
data and graphically positioned in the orientation in which 
the camera 200 was held at the time the captured image data 
60 was generated. As will be discussed in detail below, the 
shaded areas represent a subset of pixel signals within the 
image data field 419 that the auto-rotate unit 422 selects to 
be sent to the image processing unit 424 as the rotated image 
data. 

65 The auto-rotate unit 422 preferably comprises program 
instructions stored in the non-volatile memory 406 that are 
executed by the processing unit 404. The auto-rotate unit 
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422 controls the flow of data between the frame buffer 416 data field 419 location (r^, c 2 ) to the image processing unit 
and the image processing unit 424. First, the auto-rotation 424. Decrementing the row number by one, the portrait_left 
unit 422 accesses the RLOS signal 421 data field and UDOS spooling routine next transfers the pixel signal at location 
signal 423 data field from the frame buffer data structure (r £ _ 2 , c 2 ) to the image processing unit 424, and so on until 
417; and, based on this orientation data, selects either a 5 pixel (r 19 c 2 ) has been transferred. The portrait_Jeft spooling 
portrait_jeft spooling routine, a portrait_right spooling routine then increments the column by one to c 3 and resets 
routine or a landscape spooling routine. The spooling rou- the row to r M in preparation for spooling the next line of 
tines select-out and feed the captured image data stored in pixel signals (having a length of i-1) to the image processing 
the image data field 419 to the image processing unit 424. unit 424, in the same manner as the first line (having a length 
"Spooling" is here defined as a routine for transferring data 10 of i-1) was spooled. The portrait_left spooling routine 
from one unit or device to a second unit or device. It is continues to increment the column number and spool the 
through these spooling routines that the auto-rotate unit 422 next line of pixel signals (having a length of i-1) to the 
transforms the captured image data into the rotated image image processing unit 424 until column c, has been spooled, 
data. While the effect of the auto-rotate unit 422 is only The pixel signals within either row r, or column c a are not 
described below with reference to the captured image data, 15 sent to the image processing unit 424 since the image 
some of the additional information stored in the working processing unit 424 is preferably designed to process only an 
memory 418 (such as the location of any defective pixels, i-1 by j-1 array of pixel signals. As a result of this spooling 
rows or columns within the image sensor 304) will also be sequence, the entire set of pixel signals representing a 
rotated, as described above. This additional information is portrait_left image has been transferred to the image pro- 
then used by the image processing unit 424. Alternatively, a 20 cessing unit 424. Furthermore, due to this specific spooling 
defective pixel correction algorithm can be applied to the sequence, the portrait left image will be displayed on the 
image data prior to rotation of the image data. Each of the image display 100 so that the "TOP of the portrait_left 
different spooling routines are described in detail below. image corresponds to the top 102 of the image display 100. 

First, the auto-rotate unit 422 determines the image sen- If the portrait__right spooling routine is selected, the 
sor's 304 orientation at the time the image was captured, by 25 auto-rotate unit 422 interprets the captured image data 
accessing the RLOS signal 421 and UDOS signal 423 data within the image data field 419 as shown in FIG. 5B. FIG. 
fields. Next, if the RLOS signal 421 data field contains a 5B shows an i-row by j-column set of captured image data 
portrait_left signal and the UDOS signal 423 data field in a portrait_right orientation. As shown, the "TOP" of the 
contains a level signal, the auto-rotate unit 422 selects the portrait„right orientation corresponds to column c ; _ r A 
portrait_left spooling routine. However, if the RLOS signal 30 subset of the image data field 419 is selected by the 
421 data field contains a portrait_right signal and the UDOS portrait_right spooling routine, as shown by the shaded gray 
signal 423 data field contains a level signal, the auto-rotate area in FIG. 5B, to be sent to the image processing unit 424. 
unit 422 selects the portrait_right spooling routine. Lastly, The shaded area of FIG. 5B is defined from rows r 2 to r„ and 
if the RLOS signal 421 data field contains a landscape signal from columns c a to c^. The total array size of the shaded 
and the UDOS signal 423 data field contains a level signal, 35 area is therefore comprised of i-1 number of rows and j-1 
or whenever the UDOS signal 423 data field contains an number of columns. Next, the portrait_right spooling rou- 
off-level signal, regardless of what the RLOS signal 421 data tine defines the shaded area of as the top line of the 
field contains, the auto-rotate unit 422 selects the landscape rotated image data and configures the image processing unit 
spooling routine. Each of the three spooling routines within 424 for a line length of i-1 pixels. The portrait_right 
the auto-rotate unit 422 are now discussed below. 40 spooling routine first transfers the pixel signal in the image 
If the portrait_Jeft spooling routine is selected, the auto- data field 419 location (r 2 , Cy.J to the image processing unit 
rotate unit 422 interprets the captured image data within the 424. Incrementing the row number by one, the portraits 
image data field 419 as shown in FIG. 5A. FIG. 5A shows right spooling routine next transfers the pixel signal at 
an i-row by j-column set of captured image data in a location (r 3 , c^) to the image processing unit 424, and so 
portrait_Jeft orientation. As shown, the "TOP" of the 45 on until pixel fa, c,^) has been transferred. The portrait^ 
portrait_lef t orientation corresponds to column c 2 . Note that right spooling routine then decrements the column by one to 
the "row" and "column" designations defined for the image c ; _ 2 and resets the row to r 2 in preparation for spooling the 
sensor 304 in FIG. 3C are retained for clarity in FIGS. 5A, next line of pixel signals (having a length of i-1) to the 
SB and 5C even though the camera has been rotated into the image processing unit 424, in the same manner as the first 
portrait_left orientation. Conventional image processing 50 line (having a length of i-1) was spooled. The portrait_right 
units are designed to receive image data in a constant pixel spooling routine continues to decrement the column number 
pattern format. Using the Bayer pattern format (assumed for and spool the next line of pixel signals (having a length of 
the purposes of this discussion) the image processing unit i-1) to the image processing unit 424 until column c x has 
424 must receive the image data from the image data field been spooled. Thus, the entire set of pixel signals represent- 
419 starting with a "GRGRGR" line and followed by a 55 ing a portrait_right image has been transferred to the image 
"BGBGBG" line, in an alternating line-by-line pattern. processing unit 424. As a result of this specific spooling 
Therefore, a subset of the image data field 419 is selected by sequence, the portrait__right image will be displayed on the 
the portrait_left spooling routine, as shown by the shaded image display 100 so that the "TOP" of the portrait_right 
gray area in FIG. 5A, to be sent to the image processing unit image corresponds to the top 102 of the image display 100. 
424. The shaded area of FIG. 5Ais defined from rows r 2 to 60 If the landscape spooling routine is selected, the auto- 
r £ _ l9 and from columns c 2 to c ; . The total array size of the rotate unit 422 interprets the captured image data within the 
shaded area is therefore comprised of i-1 number of rows image data field 419 as shown in FIG. 5C. FIG. 5C shows 
and j-1 number of columns. Next, the portrait_left spooling an i-row by j-column set of captured image data in a 
routine defines the shaded area of c 2 as the top line of the landscape orientation. As shown, the "TOP" of the land- 
rotated image data and configures the image processing unit 65 scape orientation corresponds to r r A subset of the image 
424 to accept a line length of i-1 pixels. The portranMeft data field 419 is selected by the landscape spooling routine, 
spooling routine first transfers the pixel signal in the image as shown by the shaded gray area in FIG. 5C, to be sent to 
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the image processing unit 424. The shaded area of FIG. 5C 
is denned from rows r a to r t _ lf and from columns c 3 to c y-1 . 
The total array size of the shaded area is therefore comprised 
of i-1 number of rows and j-1 number of columns. Next, the 
landscape spooling routine defines the shaded area of t 1 as 5 
the top line of the rotated image data and configures the 
image processing unit 424 for a line length of j-1 pixels. 
Alternatively, the shaded gray area representing the rotated 
image data could be positioned over different sets of pixel 
signals within the image data field 419, again subject to the 10 
constraint that the top line begin with the "GRGRGR" 
pattern. The landscape spooling routine first transfers the 
pixel signal in the image data field 419 location (r 1? c x ) to the 
image processing unit 424. Incrementing the column num- 
ber by one, the landscape spooling routine next transfers the 15 
pixel signal at location (r 15 c 2 ) to the image processing unit 
424, and so on until pixel (r l9 c ; _ a ) has been transferred. The 
landscape spooling routine then increments the row by one 
to r 2 and resets the column to c 1 in preparation for spooling 
the next line of pixel signals (having a length of j-1) to the 
image processing unit 424, in the same manner as the first 
line (having a length of j-1) was spooled. The landscape 
spooling routine continues to increment the row number and 
spool the next line of pixel signals (having a length of j-1) 
to the image processing unit 424 until row r £ml has been 
spooled. Thus, the entire set of pixel signals representing a 
landscape image has been transferred to the image process- 
ing unit 424. As a result of this specific spooling sequence, 
the landscape image will be displayed on the image display 
100 so that the "TOP" of the landscape image corresponds 
to the top 102 of the image display 100. 

Also as introduced above, the image processing unit 424 
preferably comprises program instructions stored in the 
non-volatile memory 406 that are executed using the pro- 
cessing unit 404. The image processing unit 424 receives the 
rotated image data from the auto-rotate unit 422, performs 
conventional image processing operations (such as white 
balancing, reconstruction, color correction, gamma 
correction, sharpening and compression) on the rotated 
image data; generates processed image data; and either saves 
the processed image data to the storage memory 420 or 
outputs it to a host computer (not shown) via the I/O 
interface 414. As was discussed above, the image processing 
unit 424 is preferably implemented such that it must receive 
all the image data spooled from the frame buffer 416 starting 
with a "GRGRGR" line followed by a "BGBGBG" fine in 
an alternating line-by-line .pattern. This is the image pro- 
cessing unit's 424 design limitation which drives the auto- 
rotate unit's 422 particular pixel selection routines described 
above. Alternatively, a different image processing unit 424 
could be implemented which requires that the image data be 
spooled from the frame buffer 416 starting with a "GBG- 
BGB" line followed by a "RGRGRG" line or any combi- 
nation of the Bayer pattern. 

Those skilled in the art will also recognize that alternate 
embodiments of the present invention that do not use the 
Bayer pattern may still use the teachings of present inven- 
tion to rotate their image data. In such alternative embodi- 
ments the auto-rotate unit 422 performs a straightforward 
rotation of the image data since such alternative image 
processing units 424 may accept pixel signals in any orien- 
tation. Thus for the straightforward rotation where the 
shaded gray area of FIG. 5C represents the image data, the 
portrait_left spooling routine would start with pixel signal 
( r /-i> c iX spool column c l7 increment the column to c 2 , spool 
column c 2 and so on; the portrait_right spooling routine 
would start with pixel signal (r 1? c^), spool column c } _ l9 
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decrement the column to c y _ 2 , spool column c ; „ 2 and so on; 
and the landscape spooling routine would start with pixel 
signal (r-t, cj, spool row r 1? increment the row to r 2 , spool 
row r 2 and so on. In all three cases above, the extra or 
provided row r z and extra or provided column c ; would not 
be transferred. 

Referring now to FIGS. 6A, 6B, 6C and 6D, a flowchart 
of a preferred method for rotating the display orientation of 
a captured image is shown. The method begins in FIG. 6A 
with step 600 where the image capture unit 428 captures a 
set of image data and a set of image sensor 304 orientation 
data approximately simultaneously. The RLOS 421 and 
UDOS 423 orientation signals are recorded when the image 
data capture occurs in order to insure that the orientation 
sensor information accurately reflects the actual position of 
the image sensor 304. In step 602, the image capture unit 
428 transfers the captured image data and the image sensor 
304 orientation data into the volatile memory 410. In Step 
603, the image capture unit 428 generates an additional row 
20 and column of image data if the captured image data does 
not already include an extra row and column of image data. 
In Step 604, the auto-rotate unit 422 determines the orien- 
tation of the captured image data. If in step 606 the captured 
image data is in a portrait_left orientation, the method 
25 proceeds to step 612, otherwise the method continues to step 
608. If in step 608 the captured image data is in a portraits 
right orientation, the method proceeds to step 628, otherwise 
the method proceeds to step 644. 

In step 612 of FIG. 6B, the auto-rotate unit 422 configures 
30 the image processing unit 424 to accept an image data line 
length corresponding to a portrait_left image. In step 614, 
the auto-rotation unit 422 initializes a variable named "col- 
umn" stored in the working memory 418 to a first pixel 
column (i.e. to column cj, within the image data field 419, 
35 required by the image processing unit 424. In step 616, the 
auto-rotation unit 422 initializes a variable named "row" 
stored in the working memory 418 to a row (i.e. to row r^) 
within the image data field 419 containing a first pixel color 
required by the image processing unit 424. Next, in step 618, 
40 the auto-rotate unit 422 transfers the captured image data in 
the image data field 419 location (r wvv , c column ) to the image 
processing unit 424. In step 620, the auto-rotate unit 422 
decrements "row" to a row containing a next pixel color 
required by the image processing unit 424. In step 622, if 
45 "last row" has been transferred, the method proceeds to step 
623, otherwise the method returns to step 618. In the 
preferred embodiment "last row" is as shown in FIG. 
5A. When "last row" has been transferred in step 622, an 
entire line of portrait_Jeft image data has been transferred to 
50 the image processing unit 424. In step 623 the image 
processing unit 424 performs any required image processing 
on the transferred image data. Step 624 then increments 
"column" to a next pixel column required by the image 
processing unit 424. Finally, if in step 626 "last column" has 
55 not yet been transferred the method returns to step 616, 
otherwise the method ends. In the preferred embodiment 
"last column" is "c fi " as shown in FIG. 5 A. 

In step 628 of FIG. 6C, the auto-rotate unit 422 configures 
the image processing unit 424 to accept an image data line 
60 length corresponding to a portrait_right image. In step 630, 
the auto-rotation unit 422 initializes a variable named "col- 
umn" stored in the working memory 418 to a first pixel 
column (i.e. to column c._j), within the image data field 419, 
required by the image processing unit 424. In step 632, the 
65 auto-rotation unit 422 initializes a variable named "row" 
stored in the working memory 418 to a row (i.e. to row r 2 ) 
within the image data field 419 containing a first pixel color 



